Measure-Valued Differentiation for Markov Chains
نویسندگان
چکیده
منابع مشابه
Measure-Valued Differentiation for Markov Chains
This paper addresses the problem of sensitivity analysis for finite-horizon performance measures of general Markov chains. We derive closed-form expressions and associated unbiased gradient estimators for the derivatives of finite products of Markov kernels by measure-valued differentiation (MVD). In the MVD setting, the derivatives of Markov kernels, called D-derivatives, are defined with resp...
متن کاملMeasure-Valued Differentiation for Stationary Markov Chains
We study general state-space Markov chains that depend on a parameter, say, . Sufficient conditions are established for the stationary performance of such a Markov chain to be differentiable with respect to . Specifically, we study the case of unbounded performance functions and thereby extend the result on weak differentiability of stationary distributions of Markov chains to unbounded mapping...
متن کاملKernel Estimation for Real-valued Markov Chains
Let {Xn : n = 0, 1, 2, ...} be a real-valued Markov chain. The purpose of this paper is to study properties of kernel type estimators of the stationary density and of the transition density of such a chain. These estimators were considered first by Roussas (1969) and by Rosenblatt (1970). Both authors extended to the Markov chain case some results for kernel estimators of the density based on a...
متن کاملThree-Valued Abstraction for Continuous-Time Markov Chains
This paper proposes a novel abstraction technique for continuous-time Markov chains (CTMCs). Our technique fits within the realm of three-valued abstraction methods that have been used successfully for traditional model checking. The key idea is to apply abstraction on uniform CTMCs that are readily obtained from general CTMCs, and to abstract transition probabilities by intervals. It is shown ...
متن کاملTaylor Expansion for the Entropy Rate of Hidden Markov Chains
We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Optimization Theory and Applications
سال: 2007
ISSN: 0022-3239,1573-2878
DOI: 10.1007/s10957-007-9297-7